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DETAILED ACTION 



1 . This Office action for U.S. Patent Application No. 10/540,217 is responsive to 
communications filed 22 April 2010, in reply to the Non-Final Rejection of 25 November 
2009. Claims 3, 7, 13-16, 21, 24-26, 28, 29, 32-34, and 36^13 are pending. 

2. In the previous Office action, Claims 3, 9, 10, 13, 14, 24-29, and 38-41 were 
rejected under 35 U.S.C. § 101 as non-statutory. Claims 3, 7, 9-16, 21, and 38-41 
were rejected under 35 U.S.C. § 103(a) as obvious over U.S. Patent Application 
Publication No. 2003/0161399 A1 (AH) in view of U.S. Patent No. 6,289,049 B1 {Kim) 
and in view of U.S. Patent 7,367,042 B1 (Dakss). Claims 24-27, 29, 32-35, 37, 42, and 
43 were rejected under 35 U.S.C. § 103(a) as obvious over AH in view of Kim and Dakss 
and in view of U.S. Patent No. 6,243,495 B1 (Naveen). Claim 36 was found to contain 
obvious subject matter but was objected to as dependent on a rejected base claim. 



Response to Amendment 

3. Applicant's amendments to the claims have been fully considered but are 
insufficient to overcome the rejections under 35 U.S.C. § 101 . Applicant has amended 
the preambles of independent method claims 13, 38, and 42 to each recite that the 
respective methods are performed "using a processor" or "implemented using the 
processor". However, these amendments are not given any weight since they are in the 
claim preambles. In response to applicant's arguments, the recitation. A preamble is 
generally not accorded any patentable weight where it merely recites the purpose of a 
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process or the intended use of a structure, and where the body of the claim does not 
depend on the preamble for completeness but, instead, the process steps or structural 
limitations are able to stand alone. See In re Hirao, 535 F.2d 67, 190 USPQ 15 (CCPA 
1976) and Kropa v. Robie, 187 F.2d 150, 152, 88 USPQ 478, 481 (CCPA 1951). To 
properly overcome the rejection under 35 U.S.C. § 101 , Applicant must further amend 
the claims to recite the processor actively performing at least what Applicant considers 
to be the inventive step in the claimed methods. 



Response to Arguments 

4. Applicant's arguments, see pages 17-19, filed 22 April 2010, with respect to 
claims 13 and 15 have been fully considered and are persuasive. The prior art claim 
rejections have been withdrawn. In particular, the claims now are clearly directed to a 
method and apparatus for segmenting a motion image into different regions or objects, 
the first segmenting step comprising producing an initial estimate of which blocks 
belong in which numbered regions each with an object identification code, performing a 
second estimate of which blocks belong in which regions based on the motion between 
the current frame and a different frame at a different time, and overriding or updating the 
first estimate with the second temporal estimate. Although there is known prior art 
concerning tracking moving objects in an image based on temporal-based 
segmentation, such as in the attached document Moving Target Classification and 
Tracking from Real-time Video, the claimed method of using the motion estimate as a 
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"second estimate" for which blocks are in which numbered regions is novel-and non- 
obvious. 

5. Applicant's arguments filed with respect to Claims 38 and 39 have been fully 
considered but they are not persuasive. Applicant discusses features of the claims and 
makes a conclusory statement that the features "are new and obvious over the prior art" 
and without disclosure nor suggestion in the cited references, but presents no argument 
or evidence to support this statement. Applicant is reminded that under 37 C.F.R. § 
1.111 (b), a general allegation that the claims define a patentable invention without 
specifically pointing out how the language of the claims patentabiy distinguishes them 
from the references is not acceptable. 

6. Applicant's arguments filed with respect to Claim 42 have been fully considered 
but they are not persuasive. Applicant discusses a feature of the claim and makes a 
conclusory statement that the feature "is new and obvious over the prior art" and without 
disclosure nor suggestion in the cited references, but presents no argument or evidence 
to support this statement. Applicant is reminded that under 37 C.F.R. § 1.111 (b), a 
general allegation that the claims define a patentable invention without specifically 
pointing out how the language of the claims patentabiy distinguishes them from the 
references is not acceptable. 
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Claim Rejections - 35 USC §112 

7. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

8. Claim 16 recites the limitation "the method according to claim 15" in the first line. 
There is insufficient antecedent basis for this limitation in the claim. Claim 15 is an 
apparatus claim, not a method claim. 

Claim Rejections - 35 USC § 101 

9. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

Claims 3, 13, 14, 16, 24-26, 28, 29, 32-34, 38, and 40-42 are rejected under 35 
U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. 
The independent method claims each recite "a processor" in the claim preambles. 
Although such a recitation of such a device would tie the claimed method to a particular 
machine in a way that would satisfy the requirements of In re Bilski, 545 F.3d 943, 960 
(Fed. Cir. 2008) (aff'd on other grounds sub nom. Bilski v. Kappos, Case No. 08-964, 
Jun. 28, 2010), the presence of the "processor" in the claim preambles is not sufficient. 
A preamble is generally not accorded any patentable weight where it merely recites the 
purpose of a process or the intended use of a structure, and where the body of the 
claim does not depend on the preamble for completeness but, instead, the process 
steps or structural limitations are able to stand alone. See In re Hirao, 535 F.2d 67, 190 
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USPQ 15 (CCPA 1976) and Kropa v. Robie, 187 F.2d 150, 152, 88 USPQ 478, 481 
(CCPA 1951). It is suggested that the independent method claims be amended so that 
the "processor" actively performs at least the method step which Applicant believes to 
be the inventive step. 



Claim Rejections - 35 USC § 103 

1 0. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

1 1 . This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 

12. Claims 3, 7, 38, and 39 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over U.S. Patent Application Publication 2003/0161399 A1 (AN) in view of 
U.S. Patent 6,289,049 B1 (Kim et al.) and in view of U.S. Patent 7,367,042 B1 (Dakss et 
al.). 
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Regarding Independent Claim 38, figure 2 of AH illustrates the claimed N 
consecutive pictures within a time-series of pictures. The combined teachings of AH, 
Kim, and Dakss disclose the claimed process of step (a) of assigning the same 
identification code to adjacent blocks as needed 1 . In step (b), AH discloses determining 
a boundary of an object as a point in which two adjacent blocks have disparate motion 
vectors. AH at paragraph 0017. This is the claimed step of determining if objects having 
two different identification codes are in contact with each other. AH also discloses 
determining that two adjacent blocks are in the same object if they have similar motion 
vectors. Id. This is the claimed step of determining that correlation within a first object 
is above a predetermined value. In step (c), Dakss describes a "flood-fill" algorithm that 
allows a user to highlight a specified region in interactive viewing. Dakss column 24: 
line 50-column 26: line 51 . This process may be done two-dimensionally, in which 
every pixel in a region is given the highlight color. Id. at 24:50-56. There may also be a 
three-dimensional fill, in which the pixels in a 3D "space-time" volume may also be filled, 
with the three dimensions being horizontal, vertical, and time. Id. at 24:56-25:17. In 
other words, a specified region is tracked throughout a plurality of frames. Figures 1 1 A 
and 1 1 B illustrate this process. At block N, every pixel in a region (r, c) is given a 
highlight color. Id. at 25:18-24. Using the tracking method, both the corresponding 
pixels in the same region at frame N+1 and frame N-1 may also be highlighted. Id. at 
25:24-39. Then, the tracking of objects in Dakss may be backward, as required in 
element (c). 

1 The previous Office action, including the rejection of the prior version of Claim 13, describing the 
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Regarding Claim 3, from figure 19 and page 40, it appears that the correlation is 
a measure of relative speed of overlapped objects. For example, as shown in figure 19, 
object 1 is moving faster than object 2, and so a greater amount of it is visible from 
beneath object 2 at time t than at time t-1 . From this, it seems that the correlation 
function may be a function of relative magnitude of motion, or speed. However, in AH, 
as shown in figure 2D, two regions having "disparate" motion vectors, the foreground 
and background, have different speeds, with the foreground moving relatively quickly to 
the background. Then, the claimed algorithm is considered to be encompassed within 
the process of determining disparate motion vectors in AH. 

Regarding Independent Claim 39, AH discloses the claimed "storage device" and 
"processor". As mentioned with respect to Claim 38 supra, the combined teachings of 
AH, Kim, and Dakss, disclose step (a), AH discloses step (b), and Dakss discloses step 
(c). 

Regarding Claim 7, from figure 19 and page 40, it appears that the correlation is 
a measure of relative speed of overlapped objects. For example, as shown in figure 19, 
object 1 is moving faster than object 2, and so a greater amount of it is visible from 
beneath object 2 at time t than at time t-1 . From this, it seems that the correlation 
function may be a function of relative magnitude of motion, or speed. However, in AH, 
as shown in figure 2D, two regions having "disparate" motion vectors, the foreground 
and background, have different speeds, with the foreground moving relatively quickly to 



combination of the cited references to produce the claimed spatial segmentation and the rationale for 
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the background. Then, the claimed algorithm is considered to be encompassed within 
the process of determining disparate motion vectors in AH. 

13. Claims 24-26, 28, 29, 32-34, 37, 42, and 43 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over AH in view of Kim and in view of Dakss and in view of U.S. 
Patent 6,243,495 B1 (Naveen). Regarding Independent Claim 42, Dakss discloses the 
claimed object map, as object mapping table 217 containing the UID values for the 
different regions. Dakss at column 1 1 : lines 26-37. Then, the process of determining 
the similar motion vectors for the blocks within a region in paragraph 0017 of AH is step 
(a). The forward and backward tracking of an object of interest in figure 1 1 A and 
column 25: lines 19-38 of Dakss is the process of determining the region based on the 
motion vector and an object map as the region moves forward or backward in time in 
step (b). However, none of the cited references disclose the claimed step of using a 
weighted average of motion vectors with weight as a function of overlap. In AH, a 
normal forward motion vector from the figure 2B frame at time t1 to the figure 2C frame 
at time 2 is the claimed "fast-forward motion vector". As previously mentioned, in the 
combined teachings of AH and Kim, a pair of adjacent motion vectors having an 
absolute value difference greater than a threshold is considered to be a pair of 
"disparate" motion vectors in different regions according to paragraph 0017 of AH. 

Naveen teaches a method of segmenting or partitioning MPEG video. Regarding 
Claim 42, in Naveen, motion vectors are first obtained according to standard procedure. 



combination required under KSR v. Teleflex, is incorporated by reference. 
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Next, the motion vectors are refined according to the weighted average of surrounding 
macroblocks, with the weight proportional to the overlap of the area of the surrounding 
motion-compensated macroblocks. Naveen, column 4: lines 42>-41, column 6: lines 5- 
10. This is the claimed step of using a weighted motion vector average with the weight 
corresponding to the areas of overlapping blocks and the region of interest, or current 
block. 

Ali, in combination with Kim and Dakss, discloses the claimed invention except 
for the weighted motion vectors proportional to overlap area. Naveen discloses using 
this weighing. Therefore, it would have been obvious to one having ordinary skill in the 
art at the time of the present invention to modify the system of Ali to use the "refined" 
motion vectors of Naveen, since Naveen states in column 2: lines 39-44 that such a 
modification would enable the use of more high quality motion coding, particularly for 
video that is recorded or encoded at real-time but does not need to be played back live. 

Regarding Claim 24, in Ali, motion vector determination is done blockwise. Ali at 
paragraph 001 7. Then, the "region of interest" of step (a) of the parent claim that is the 
basis of the motion vector corresponds to one block, as required in claim 24. 

Regarding Claim 25, in Dakss, a new mask containing the region maps is 
generated for each frame. Dakss, column 10: lines 62-64. Then, Dakss generates a 
new map for each frame. The motion backwards from frame N to frame N-1 in figure 
1 1A is the claimed movement in the negative direction of the motion vector. 
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Regarding Claim 26, in Dakss, since a new mask is generated for each frame, it 
must always update the oldest object maps with a newest object map, as claimed. 

Regarding Claim 27, in AH, a normal forward motion vector from the figure 2B 
frame at time t1 to the figure 2C frame at time 2 is the claimed "fast-forward motion 
vector". As previously mentioned, in the combined teachings of AH and Kim, a pair of 
adjacent motion vectors having an absolute value difference greater than a threshold is 
considered to be a pair of "disparate" motion vectors in different regions according to 
paragraph 0017 of AH. 

Regarding Claim 29, as discussed with respect to for example claim 9 supra, in 
AH and Kim, the process of segmenting a picture into objects or regions based on the 
absolute value of difference of motion vectors is the claimed process of determining of 
the absolute value of the difference is more than a threshold value. Two adjacent 
frames are at time t1 and t2, with the "interval of time" as the temporal distance between 
the two adjacent frames. 

Regarding Independent Claim 43, as mentioned with respect to Claim 15 supra, 
AH discloses the claimed "storage device" and "processor". As mentioned with respect 
to Claim 42 supra, the combined teachings of AH and Dakss disclose the process of 
steps (a) and (b) and Naveen discloses the claimed process of using the weighted 
motion vectors. 



Application/Control Number: 10/540,217 Page 12 

Art Unit: 2621 

Regarding Claim 32, in AH, motion vector determination is done blockwise. AH at 
paragraph 0017. Then, the "region of interest" of step (a) of the parent claim that is the 
basis of the motion vector corresponds to one block, as required in claim 32. 

Regarding Claim 33, in Dakss, a new mask containing the region maps is 
generated for each frame. Dakss, column 10: lines 62-64. Then, Dakss generates a 
new map for each frame. The motion backwards from frame N to frame N-1 in figure 
1 1 A is the claimed movement in the negative direction of the motion vector. 

Regarding Claim 34, in Dakss, since a new mask is generated for each frame, it 
must always update the oldest object maps with a newest object map, as claimed. 

Regarding Claim 37, as discussed with respect to for example claim 9 supra, in 
AH and Kim, the process of segmenting a picture into objects or regions based on the 
absolute value of difference of motion vectors is the claimed process of determining of 
the absolute value of the difference is more than a threshold value. Two adjacent 
frames are at time t1 and t2, with the "interval of time" as the temporal distance between 
the two adjacent frames. 

Allowable Subject Matter 

14. Claim 15 is allowed. 

15. Claims 13, 14, 21, 40, and 41 would be allowable upon actions that would 
obviate the rejection under 35 U.S.C. § 101. 

16. Claim 16 would be allowable upon actions that would obviate the rejections 
under 35 U.S.C. §§ 101 and 112. 
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17. Claim 36 is objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. 

18. The following is an examiner's statement of reasons for allowance: As Applicant 
mentioned in the Remarks and for the reasons discussed above in the "Response to 
Arguments" section, Claim 15 is directed to an apparatus that performs a novel and 
non-obvious video segmentation. 

Any comments considered necessary by applicant must be submitted no later 
than the payment of the issue fee and, to avoid processing delays, should preferably 
accompany the issue fee. Such submissions should be clearly labeled "Comments on 
Statement of Reasons for Allowance". 

1 9. The following is a statement of reasons for the indication of allowable subject 
matter: Claim 36 is directed to a novel and non-obvious process of segmenting images 
based on clustering histograms of motion vector speeds of blocks and detecting 
different peaks in the histograms, the peaks corresponding with different regions, as 
illustrated in figure 26. 

Conclusion 

20. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. U.S. Patent Application Publication No. 2003/0086490 A1 {Horl) 
teaches a video segmentation system that updates which "reference object region" is 
used as the temporal reference for a region in a current frame. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David N. Werner whose telephone number is (571 )272- 
9662. The examiner can normally be reached on Monday-Friday from 8:30 to 5:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on (571) 272-7418. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



ID. N. W./ 

Examiner, Art Unit 2621 
/Mehrdad Dastouri/ 

Supervisory Patent Examiner, Art Unit 2621 



